Optronic viewing device for a land vehicle

ABSTRACT

An optronic vision apparatus with which a land vehicle is intended to be equipped, includes a panoramic image sensor, at least one orientable camera, having a better resolution in a field of view that is smaller than the field of view of the panoramic image sensor, and an image-displaying device; wherein it also comprises a data processor that is configured or programmed to: receive at least one first image from the panoramic image sensor and one second image from the orientable camera; from the first and second images, synthesize a composite image in which at least one section of the second image is embedded in a section of the first image; and transmit the composite image to the image-displaying device. An armored vehicle equipped with such an optronic vision apparatus. A method by such an optronic apparatus is provided.

The invention relates to an optronic vision apparatus for a landvehicle, in particular an armored vehicle or a tank.

It is known to equip such a vehicle with a plurality ofdetecting/designating cameras that operate in the visible or in theinfrared, said cameras being orientable and having a field of view of afew degrees—typically variable between 3° and 9° or between 4° and 12°,and possibly able to reach as much as 20°. These cameras have a verygood resolution, but use thereof is not easy because of their smallfield: it is difficult for the operator to locate the small regionobserved by such a camera in the environment through which the vehicleis moving. In this respect, the “straw effect” is spoken of, because itis as though the operator were looking through a straw.

It is possible to mitigate this drawback by modifying the optics ofthese cameras so as to allow them to operate in a very-large-field mode(field as large as 40°-45°). It is expensive to implement such asolution and, in addition, switching the camera to very-large-field modeprevents simultaneous small-field vision.

Document U.S. 2002/75258 describes a surveillance system comprising apanoramic first camera with a plurality of lenses, and an orientablehigh-resolution second camera. In this system, a high-resolution imageacquired by the second camera is embedded into a panoramic image issuedfrom the first camera.

The invention aims to overcome the drawbacks of the prior art, and toprovide a vision system that is better suited to the requirements of thecrew of a vehicle such as an AFV or tank.

To do this, it exploits the fact that modern armored vehicles are oftenequipped with a very-large-field vision system, for example ahemispherical sensor such as the “ANTARES” system from Thales, whichenables vision over an azimuthal angle of 360° and along a vertical arcof −15° to 75°. This very large field of view includes that of the oneor more detecting/designating cameras, at least for certain ranges oforientation of the latter. An idea on which the invention is based thusconsists in combining, in a given display, an image section acquired bysuch a very-large-field vision system and an image acquired by adetecting/designating camera. According to the invention, thehigh-resolution small-field image delivered by the detecting/designatingcamera is embedded into a section of a lower-resolution larger-fieldimage issued from the vision system. Thus a synthesis imagecorresponding to the image that would be acquired by a virtual camerahaving the same orientation as the detecting/designating camera, but alarger field of view, is obtained. Advantageously, the user may zoom in,in order to exploit the high-resolution of the detecting/designatingcamera, or zoom out, in order to increase his field of view and, forexample, identify reference points.

Thus, one subject of the invention is an optronic vision apparatus withwhich a land vehicle is intended to be equipped, comprising:

-   -   a panoramic image sensor;    -   at least one orientable camera, having a better resolution in a        field of view that is smaller than the field of view of the        panoramic image sensor and that is contained in the latter field        of view for at least one set of orientations of the camera; and    -   an image-displaying device;

also comprising a data processor that is configured or programmed to:

-   -   receive at least one first image from said panoramic image        sensor and one second image from said orientable camera;    -   from the first and second images, synthesize a composite image        in which at least one section of the second image is embedded in        a section of the first image; and    -   transmit said composite image to the image-displaying device;        the data processor being configured or programmed to synthesize        said composite image such that it corresponds to an image that        would be acquired by a virtual camera having the same        orientation as said orientable camera, but a larger field of        view.

According to particular embodiments of such an apparatus:

The data processor may be configured or programmed to modify the size ofthe field of view of said composite image in response to a commandoriginating from a user.

The data processor may be configured or programmed to synthesize inreal-time a stream of said composite images from a stream of said firstimages and a stream of said second images.

The image-displaying device may be a portable displaying device equippedwith orientation sensors, the apparatus also comprising a system forservo-controlling the orientation of the camera to that of the portabledisplaying device.

The panoramic image sensor and the orientable camera may be designed tooperate in different spectral ranges.

The panoramic image sensor may be a hemispherical sensor.

The orientable camera may have a possibly variable field of viewcomprised between 1° and 20° and preferably between 3° and 12°.

Another subject of the invention is an armored vehicle equipped withsuch an optronic vision apparatus.

Yet another subject of the invention is a method implemented by such anoptronic apparatus, comprising the following steps:

-   -   receiving a first image from a panoramic image sensor;    -   receiving a second image from an orientable camera, said second        image having a better resolution in a field of view that is        smaller than the field of view of the first image and that is        contained in the latter field of view;    -   from the first and second images, synthesizing a composite image        in which at least one section of the second image is embedded in        a section of the first image, said composite image corresponding        to an image that would be acquired by a virtual camera having        the same orientation as said orientable camera, but a larger        field of view; and    -   displaying said composite image.

According to particular embodiments of such a method:

The method may also comprise the following step: modifying the size ofthe field of view of said composite image in response to a commandoriginating from a user.

A stream of said composite images may be synthesized in real-time from astream of said first images and a stream of said second images.

Said composite image may be displayed on a portable displaying deviceequipped with orientation sensors, the method also comprising thefollowing steps: determining the orientation of said portable displayingdevice from signals generated by said sensors; and servo-controlling theorientation of the camera to that of the portable displaying device.

Other features, details and advantages of the invention will becomeapparent on reading the description given with reference to the appendeddrawings, which are given by way of example and in which:

FIG. 1 is a schematic representation of the principle of the invention;

FIGS. 2A to 2E illustrate various images displayed during animplementation of the invention; and

FIGS. 3 and 4 schematically illustrate two optronic apparatusesaccording to respective embodiments of the invention.

In this document:

The expression “detecting/designating camera” indicates an orientabledigital camera with a relatively small field of view—typically smallerthan or equal to 12° or even 15°, but possibly sometimes being as muchas 20°, both in the azimuthal plane and along a vertical arc. Adetecting/designating camera may operate in the visible spectrum, in thenear infrared (night-time vision), in the mid or far infrared (thermalcamera), or indeed be multispectral or even hyperspectral.

The expressions “very-large-field” and “panoramic” are considered to beequivalent and to designate a field of view extending at least 45° inthe azimuthal plane, along a vertical arc or both.

The expression “hemispherical sensor” designates an image sensor havinga field of view extending 360° in the azimuthal plane and at least 45°along a vertical arc. It may be a question of a single sensor, forexample one using a fisheye objective, or indeed a composite sensorconsisting of a set of cameras of smaller field of view and a digitalprocessor that combines the images acquired by these cameras. Ahemispherical sensor is a particular type of panoramic, orvery-large-field, image sensor.

As was explained above, one aspect of the invention consists incombining, in a given display, an image section acquired by ahemispherical vision system (or more generally by a very-large-fieldvision system) and an image acquired by a detecting/designating camera.This leads to the synthesis of one or more composite images that aredisplayed by means of one or more displaying devices, such as screens,virtual reality headsets, etc.

When using an apparatus according to the invention, an operator may, forexample, select a large-field viewing mode—say a field of 20° (in theazimuthal plane)×15° (along a vertical arc). The selection is performedwith a suitable interface tool: a keyboard, a thumbwheel, a joystick,etc. A suitably programmed data processor then selects a section of animage, issued from the hemispherical vision system, having the desiredfield size and oriented in the sighting direction of thedetecting/designating camera. The image acquired by thedetecting/designating camera—which for example corresponds to a field of9°×6° size, is embedded into the center of this image, with the samemagnification. This is illustrated by the left-hand panel of FIG. 1, inwhich the reference 100 designates the composite image, 101 correspondsto the exterior portion of this composite image, which originates fromthe hemispherical vision system and which provides a “context” allowingthe operator to get his bearings, whereas 102 designates its centralportion, which originates from the detecting/designating camera. It willbe noted that the elementary images 101 and 102 have the samemagnification, there is therefore no break in continuity in the scenedisplayed by the composite image. Leaving aside the fact that theelementary images 101 and 102 do not have the same resolution and maycorrespond to separate spectral ranges, and that parallax is hard toavoid, the composite image corresponds to the image that would beacquired by a camera—which could be qualified a “virtual” camera—havingthe same orientation as the detecting/designating camera but a largerfield of view.

Contrary to the case of the surveillance system of document U.S.2002/75258, when the orientation of the detecting/designating camera ismodified the elementary image 102 does not move in the interior of theelementary image 101. In contrast, the field of view of the latter ismodified to keep the elementary image 102 aligned with its centralportion. In this way, it is always the central portion of the compositeimage that has a high resolution.

If the operator zooms out, thereby further increasing the size of thefield of view, the central portion 102 of the image shrinks. If he zoomsin, this central portion 102 gets larger to the detriment of theexterior portion 101, this being shown in the central panel of FIG. 1,until said exterior portion disappears (right-hand panel in the figure).Reference is here being made to a digital zoom functionality,implemented by modifying the display. The detecting/designating cameramay also comprise an optical zoom; when the camera is optically zoomedin however, the field size of the image issued from the cameradecreases, and therefore the central portion 102 of the composite image100 shrinks. A digital zoom applied to the composite image 100 may,where appropriate, restore the dimensional ratio between the images 101and 102.

Advantageously, the detecting/designating camera and the hemisphericalvision system deliver image streams at a rate of several frames persecond. Preferably, these images are combined in real-time or almostreal-time, i.e. with a latency not exceeding 1 second and preferably0.02 seconds (the latter value corresponding to the standard duration ofa frame).

FIGS. 2A to 2E illustrate in greater detail the composite imagesdisplayed on a screen 2 of an optronic apparatus according to theinvention.

FIG. 2A shows an overview of this screen.

A strip 20 in the bottom portion of the screen corresponds to a firstcomposite image obtained by combining the panoramic image 201 (360° inthe azimuthal plane, from −15° to +75° perpendicular to this plane)issued from the hemispherical sensor and an image 202 issued from adetecting/designating camera, which in this case is an infrared camera.In fact, a rectangle of the panoramic image 201 is replaced by the image202 (or, in certain cases, by a section of this image). If thedetecting/designating camera is equipped with an optical zoom, the image202 may have a variable size, but in any case it will occupy only asmall portion of the panoramic image 201. FIG. 2B shows a detail of thisstrip. It will be noted that, in the case of a strip display, thecomposite image is not necessarily centered on the sighting direction ofthe detecting/designating camera.

The top portion 21 of the screen displays a second composite image 210showing the image 202 issued from the detecting/designating cameraembedded in a context 2011 issued from the hemispherical image sensor,in other words a section 2011 of the panoramic image 201. The user maydecide to activate a digital zoom (independent of the optical zoom ofthe detecting/designating camera) in order to decrease the size of thefield of view of the composite image 210. The embedded image 202therefore appears enlarged, to the detriment of the section 2011 of thepanoramic image. FIGS. 2C, 2D and 2E correspond to higher and higherzoom levels. FIG. 2E, in particular, corresponds to the limiting case inwhich only the image 202 issued from the detecting/designating camera isdisplayed (see the right-hand panel of FIG. 1). Of course, the user mayat any moment decide to zoom out in order to once again see the contextof the image.

One advantage of the invention is to make it possible to benefit bothfrom the strip display 20 with identification of the region observed bythe detecting/designating camera and the composite image 21 ofintermediate field size. This would not be possible if an optical zoomof the detecting/designating camera were used in isolation.

Another portion of the screen may be used to display detailed views ofthe panoramic image 201. This is without direct relationship to theinvention.

FIG. 3 schematically illustrates an optronic apparatus according to afirst embodiment of the invention, installed on an armored vehicle 3000.The reference 300 designates the hemispherical image sensor mounted onthe roof of the vehicle; 310 corresponds to the detecting/designatingcamera, installed in a turret; 311 to a joystick-type control devicethat allows an operator 350 to control the orientation of the camera 310and its optical zoom; 320 designates the data processor that receivesimage streams from the sensor 300 and from the camera 310 and thatsynthesizes one or more composite images that are displayed on thescreen 330. The bottom portion of this figure shows these compositeimages 210, which images were described above with reference to FIGS.2A-2E.

FIG. 4 schematically illustrates an optronic apparatus according to asecond embodiment of the invention, also installed on the armoredvehicle 3000. This apparatus differs from that of FIG. 3 in that thedisplay screen and the control device 311 have been replaced by aportable displaying device 430, for example resembling a pair ofbinoculars, a virtual reality headset or a tablet. This device 430 isequipped with position and orientation sensors: gyroscopes,accelerometers, optical sensors, etc. allowing its position and aboveall its orientation with respect to a coordinate system attached to thevehicle to be determined. A servo-control system 435—certain componentsof which may be shared with the data processor 320—servo-controls theorientation of the camera 310 to that of the device 430. The compositeimages displayed by the device are adapted in real-time to variations inorientation. Thus, the operator 350 is given the impression that he islooking through the armor of the vehicle 3000 with a pair of zoombinoculars. In addition, a screen may be present that displays, forexample, a very-large-field composite image such as the strip 20illustrated in FIG. 2A.

In the embodiments that have just been described, the optronic apparatuscomprises a single very-large-field vision system (a hemisphericalsensor) and a single detecting/designating camera. However, moregenerally, such an apparatus may comprise a plurality ofvery-large-field vision systems, for example operating in differentspectral regions, and/or a plurality of detecting/designating camerasthat are able to be oriented, optionally independently. Thus, aplurality of different composite images may be generated and displayed.

The data processor may be a generic computer or a microprocessor boardspecialized in image processing, or even a dedicated digital electroniccircuit. To implement the invention, it executes image-processingalgorithms that are known per se.

1. An optronic vision apparatus with which a land vehicle is intended to be equipped, comprising: a panoramic image sensor; at least one orientable camera, having a better resolution in a field of view that is smaller than the field of view of the panoramic image sensor and that is contained in the latter field of view for at least one set of orientations of the camera; and an image-displaying device; also comprising a data processor that is configured or programmed to: receive at least one first image from said panoramic image sensor and one second image from said orientable camera; from the first and second images, synthesize a composite image in which at least one section of the second image is embedded in a section of the first image; and transmit said composite image to the image-displaying device; wherein the data processor is configured or programmed to synthesize said composite image such that it corresponds to an image that would be acquired by a virtual camera having the same orientation as said orientable camera, but a larger field of view.
 2. The optronic vision apparatus as claimed in claim 1, wherein the data processor is configured or programmed to modify the size of the field of view of said composite image in response to a command originating from a user.
 3. The optronic vision apparatus as claimed in claim 1, wherein the data processor is configured or programmed to synthesize in real-time a stream of said composite images from a stream of said first images and a stream of said second images.
 4. The optronic vision apparatus as claimed in claim 1, wherein the image-displaying device is a portable displaying device equipped with orientation sensors, the apparatus also comprising a system for servo-controlling the orientation of the camera to that of the portable displaying device.
 5. The optronic vision apparatus as claimed in claim 1, wherein the panoramic image sensor and the orientable camera are designed to operate in different spectral ranges.
 6. The optronic vision apparatus as claimed in claim 1, wherein the panoramic image sensor is a hemispherical sensor.
 7. The optronic vision apparatus as claimed in claim 1, wherein the orientable camera has a possibly variable field of view comprised between 1° and 20° and preferably between 3° and 12°.
 8. An armored vehicle equipped with an optronic vision apparatus as claimed in claim
 1. 9. A method implemented by an optronic apparatus as claimed in claim 1, comprising the following steps: receiving a first image from a panoramic image sensor; receiving a second image from an orientable camera, said second image having a better resolution in a field of view that is smaller than the field of view of the first image and that is contained in the latter field of view; from the first and second images, synthesizing a composite image in which at least one section of the second image is embedded in a section of the first image, said composite image corresponding to an image that would be acquired by a virtual camera having the same orientation as said orientable camera, but a larger field of view; and displaying said composite image.
 10. The method as claimed in claim 9, wherein said composite image corresponds to an image that would be acquired by a virtual camera having the same orientation as said orientable camera, but a larger field of view.
 11. The method as claimed in claim 9, also comprising the following step: modifying the size of the field of view of said composite image in response to a command originating from a user.
 12. The method as claimed in claim 9, wherein a stream of said composite images is synthesized in real-time from a stream of said first images and a stream of said second images.
 13. The method as claimed in claim 9, wherein said composite image is displayed on a portable displaying device equipped with orientation sensors, the method also comprising the following steps: determining the orientation of said portable displaying device from signals generated by said sensors; and servo-controlling the orientation of the camera to that of the portable displaying device. 